In the past, individual applications have implemented search features independently. For example, a search in email, a search in a file system, a search in code files, etc. The search system is becoming a system-level service that different applications can use to store their data and make it searchable. Search indexes are becoming foundation services provided on a desktop, as part of the operating system of personal computers, or enterprise servers.
A specialized search facility can be provided on a desktop of a personal computer. An example is Google Desktop Search (trade mark of Google Inc.) which provides searches and access to information on a personal computer. It is a desktop search application that provides full text search over email, instant messages, computer files, media files and documents such as music, video and photos, and web pages that have been viewed.
System-wide search engines can also be provided as part of an operating system of a computer. For example, Spotlight in Mac OS X v10.4 Tiger (Spotlight, Mac and Tiger are trade marks of Apple Computer Inc.) and Longhom (trade mark of Microsoft Corporation) provide search facilities as part of their operating systems in which applications and documents can be searched.
Applications running on such systems do not implement their search features independently, as done in the past but, instead, index their data as part of the system-wide index. For example, an instant messaging application that wants to provide a searchable history of messages does not need to implement this feature but must push the messages to the global index provided by the operating system or by the specialized search service. In order to do so, providers of search technology publish APIs that allow applications to push their data into their index (e.g., Google Desktop Search API).
This approach poses the problem of several applications pushing updates to the search index concurrently and competing for resources. These different applications may have different policies regarding the speed at which their updates must appear in the index. For example, an email application may need every message to be indexed as soon as it arrives (a few seconds) while a file system search application may support a lag of a few minutes (and even more for very large file systems). Furthermore, not knowing the expectations of the applications that push data in the index, the search index itself cannot allocate resources ahead of time in order to satisfy those expectations.